On the Limiting Distribution of Lempel-Ziv'78 Redundancy for Memoryless Sources

We study the Lempel-Ziv’78 algorithm and show that its (normalized) redundancy rate tends to a Gaussian distribution for memoryless sources. We accomplish it by extending findings from our 1995 paper [4], in particular, by presenting a new simplified proof of the Central Limit Theorem (CLT) for the number of phrases in the LZ’78 algorithm. As in [4], we first analyze the asymptotic behavior of the total path length in the associated digital search tree (a DST) built from independent sequences. Then a renewal theory type argument yields CLT for LZ’78 scheme. Here, we extend our analysis of LZ’78 algorithm to present new results on the convergence of moments, moderate and large deviations, and CLT for the (normalized) redundancy. In particular we confirm that the average redundancy rate decays as 1 logn , and we find that the variance is of order 1 n where n is the length of the text.